Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: add integration tests with python 3.13 #336

Closed
wants to merge 5 commits into from

Conversation

fruch
Copy link

@fruch fruch commented Jun 20, 2024

since we want to start test the alpha/beta version of python refactoring a bit the integration test workflow

the action for pyenv we are using isn't really getting updates too much, and trying to switch back to the offical use python action that can now have prerelease python versions

also move to test newer scylla version which are not depended on python2 anymore, which was the main reason for using pyenv

@fruch fruch force-pushed the int_tests_on_3.13 branch 2 times, most recently from 1ec3337 to 6926e6b Compare July 1, 2024 20:21
@fruch
Copy link
Author

fruch commented Jul 1, 2024

7 failing tests, cause we move to test scylla 5.4:

2024-07-01T20:50:07.3045371Z FAILED tests/integration/standard/test_cluster.py::ClusterTests::test_invalid_protocol_negotation
2024-07-01T20:50:07.3049333Z FAILED tests/integration/standard/test_metadata.py::SchemaMetadataTests::test_collection_indexes - AssertionError: '(full(b))' not found in "CREATE TABLE schemametadatatests.test_collection_indexes (\n    a int PRIMARY KEY,\n    b frozen<map<text, text>>\n) WITH bloom_filter_fp_chance = 0.01\n    AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}\n    AND comment = ''\n    AND compaction = {'class': 'SizeTieredCompactionStrategy'}\n    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}\n    AND crc_check_chance = 1.0\n    AND dclocal_read_repair_chance = 0.0\n    AND default_time_to_live = 0\n    AND gc_grace_seconds = 864000\n    AND max_index_interval = 2048\n    AND memtable_flush_period_in_ms = 0\n    AND min_index_interval = 128\n    AND read_repair_chance = 0.0\n    AND speculative_retry = '99.0PERCENTILE';\nCREATE INDEX index3 ON schemametadatatests.test_collection_indexes (b);"
2024-07-01T20:50:07.3049675Z FAILED tests/integration/standard/test_metadata.py::TestCodeCoverage::test_case_sensitivity
2024-07-01T20:50:07.3051181Z FAILED tests/integration/cqlengine/management/test_compaction_settings.py::AlterTableTest::test_compaction_not_altered_without_changes_sizetiered - cassandra.protocol.ConfigurationException: <Error from server: code=2300 [Query invalid because of configuration issue] message="bucket_low value (10) must be between 0.0 and 1.0">
2024-07-01T20:50:07.3051800Z FAILED tests/integration/cqlengine/management/test_management.py::IndexTests::test_sync_indexed_set - AssertionError: unexpectedly None
2024-07-01T20:50:07.3053155Z FAILED tests/integration/cqlengine/query/test_queryset.py::TestQuerySetOrdering::test_order_by_success_case - cassandra.InvalidRequest: Error from server: code=2200 [Invalid query] message="Order by currently only supports the ordering of columns following their declared order in the PRIMARY KEY"
2024-07-01T20:50:07.3054585Z FAILED tests/integration/cqlengine/test_connections.py::TestQuerySetOrderingNoDefault::test_order_by_success_case - cassandra.InvalidRequest: Error from server: code=2200 [Invalid query] message="Order by currently only supports the ordering of columns following their declared order in the PRIMARY KEY"
2024-07-01T20:50:07.3054909Z = 7 failed, 895 passed, 118 skipped, 7 xfailed, 508 warnings in 1492.89s (0:24:52) =

if we'll move to scylla 6.0, it would be worse.

the driver matrix should have helped knowing about those, but matrix isn't running cqlengine tests... we introduced it here in the repo.
but not on the matrix, and most of those failures are in cqlengine.

@sylwiaszunejko
Copy link
Collaborator

FYI tests/integration/standard/test_metadata.py::SchemaMetadataTests::test_collection_indexes failure is due to the bug in scylla scylladb/scylladb#19278

@fruch
Copy link
Author

fruch commented Jul 2, 2024

FYI tests/integration/standard/test_metadata.py::SchemaMetadataTests::test_collection_indexes failure is due to the bug in scylla scylladb/scylladb#19278

Thanks, we should mark it in the code to be skipped somehow

cqlengine issues, I hate it, but we have customers using that, and we need to attend to it somehow.

@fruch
Copy link
Author

fruch commented Jul 2, 2024

I've found that 2 of the test are XPASS, and we are in strict mode, hence this is failing.

seems like those two now works from version 5.2 and up

fruch added 2 commits July 2, 2024 22:19
since we want to start test the alpha/beta version of python
refactoring a bit the integration test workflow

the action for pyenv we are using isn't really getting updates
too much, and trying to switch back to the offical use python action
that can now have prerelease python versions
so far the code assumed view can be name like:
```python
possible_names = [name, f'values({name})']
```

we have now one more option, that need to be considered:
```python
f'keys({name})'
```
@fruch
Copy link
Author

fruch commented Jul 2, 2024

I've handle all the issue beside

fruch added 2 commits July 2, 2024 22:50
some test that was marked with @xfail_scylla, are now passing
with newer release, i.e. some issue were fixed.

so this new decorator can xfail up to a certion scylla_version

note that it support both a oss version, and enterprise version
at some point scylla started validation of the arguments to compaction
and newer versions rejects the values passed from this test

chnaged it to something within the values expected
test is failing with the following:
```
cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
message="Order by currently only supports the ordering of columns following their declared order in the PRIMARY KEY"
```

seems like scylla doesn't support ordering by clustering keys

Ref: scylladb#343
@fruch
Copy link
Author

fruch commented Jul 5, 2024

@Lorak-mmk care to give it a look

The only failure we have here is one integration run being stuck
It's not new, we see it since we started running on multiple event loops
Still not sure what the cause

We have here changes that moves us to using scylla 5.4
And stop using pyenv in CI

cassandra/cqlengine/management.py Show resolved Hide resolved
@@ -390,6 +390,17 @@ def _id_and_mark(f):
requires_custom_payload = pytest.mark.skipif(SCYLLA_VERSION is not None or PROTOCOL_VERSION < 4,
reason='Scylla does not support custom payloads. Cassandra requires native protocol v4.0+')
xfail_scylla = lambda reason, *args, **kwargs: pytest.mark.xfail(SCYLLA_VERSION is not None, reason=reason, *args, **kwargs)


def xfail_scylla_version(reason, oss_scylla_version, ent_scylla_version, *args, **kwargs):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe xfail_scylla_version_less_than? Current name doesn't properly convey the meaning.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I thought maybe to support both options somehow, but I didn't had time todo so.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think of putting the version into pytest_markeval_namespace so can use pytest.mark.xfail directly

@Lorak-mmk
Copy link

@dkropachev I didn't yet read your new PR ( #363 ). Is there something that this PR does that yours doesn't? If not, can this be closed as succeeded by #363 >

@dkropachev
Copy link
Collaborator

Closed in favor of #363

@dkropachev dkropachev closed this Aug 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants